The field of the invention relates generally to methods and systems that provide detailed information relating to the identification of security and networking issues on Internet Protocol (IP) networks, and more specifically, to the providing of network information for manual and/or automated network analysis through packet header collection and storage.
In an IP network there is significant danger of loss of information, loss of intellectual property, theft of corporate data, inappropriate connections, or permission access violations. Locating these types of activities in a complex and busy network is extremely difficult.
As is known in the art, IP network communication generally involves the transfer of packets of data from one address and port, to another. For example, when a user wants to connect to a website through a computer's internet browser, the user's computer sends a literal Transfer Control Protocol Synchronize (TCP SYN) packet to the website's computer IP address on port 80. The website's computer then acknowledges receipt of the SYN packet, a request to open communication, and responds with a Synchronize/Acknowledge (SYN/ACK) packet to establish connection with the user's computer. The user's computer receives this grant of request for communication and responds with an ACK packet indicating to the website computer that the transfer of data can begin.
In network management and security analysis there is a need to perform detailed analysis on these packets of data, and more specifically the packet headers, that are currently traversing the network. There is also a need to view the past traffic. This monitoring may be done utilizing packet header data. However, this monitoring has proved to be difficult because there may be billions of packets per day or more traversing the network. Therefore, it is unfeasible to simply store and analyze all the packet headers in a database without extensive hardware and analyst time expense.
At least one known program attempts to store mass quantities of packet header data, allotting a set number of bytes per captured packet regardless of the amount of memory needed (length of the packet header) for the data collected. Therefore, the size of the individual capture files can either be larger than necessary, or they may be insufficient and result in the truncation of valuable information. All of the captured packets are based on an optional predetermined filter requirement and are stored separately. Therefore, even a continuous string of like-kind packet headers are represented individually in a capture file, maintaining multiple entries of nearly identical information, utilizing valuable storage space and analysis time.
At least one known network monitoring tool performs network communication tracking. However, these monitoring tools only use subsets of data and generally perform off-line, one-time analysis instead of continuous on-line analysis. Additionally these tools only track individual TCP (connection oriented) or UDP (non-connection oriented) protocol sessions instead of all sessions in a specified time interval, and only a listing of which ports were in use is indicated. Each user connection is listed independently instead of grouping the connections based on linking conversing participants (a “conversation”). Also, in only listing the ports used, there is no determination of the actual service port being used between the participants.
As is known in the art, signature-based intrusion detection and scanner detection can identify suspicious activity while an attacker is either scanning a significant number of hosts/ports in the network or sending packets that have well known characteristics or signature matches. But if an attacker only sends a small number of packets to key hosts, and the packets do not contain strings that trigger signature-based intrusion detection systems, the existing detectors will not alert the analyst to the attacker's activity. The existing detectors do not identify patterns or anomalies in network behavior that indicate a potential attack.
When an attacker is penetrating a network, it is often difficult to know exactly what the attacker did during and after the penetration. Currently the method of acquiring this type of information is for the analyst to manually go back through the log files (or logged events) captured at the time of the attack if they are even available. Reviewing log files (or logged events) manually is time consuming, is prone to user error, and is difficult or impossible to perform in real-time.
At least one known network system hosts multicast video and audio streams between various sites around the country. However, in this network system there is no capability to automatically and passively identify whether the multicast packets were transmitted by the server and actually arrived at the client (receiver). This results in wasting valuable analyst time and/or additional expense when problems occur in the transmission and/or reception of these multicast packets.